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ABSTRACT 



The Interdisciplinary Teamed Instruction (ITI) project 
investigated the effects of interdisciplinary, teamed instruction on 
secondary school teaching and learning. It examined the effectiveness of a 
professional development model that facilitated the development, 
implementation, and evaluation of ITI. Through summer institutes and onsite 
workshops, project staff provided training, resources, and ongoing technical 
assistance to teams of teachers and administrators. Teams designed and tested 
integrated curricula that incorporated learning activities and alternative 
assessments; conducted collaborative action research to determine the 
effectiveness of units; and redesigned units using feedback from key 
stakeholders. Three summer institutes were held to help teacher-administrator 
teams build capacity for collaboration and interdisciplinary teamed 
instruction. One institute helped teams design integrated units, plan 
learning activities and assessments, and present their units. Researchers 
evaluated the institute's processes, outcomes, and products via 
questionnaires, observations, interviews, reflective assignments, feedback 
forms, oral evaluation feedback, and ratings of participants' presentations. 
Overall, the summer institute affected teacher self-efficacy, decision 
making, impact, and autonomy in scheduling. Its greatest impact was on 
collaboration, cooperative learning, assessment rubrics, multiple 
intelligences, and team building. Participants were very positive about the 
institute, which strengthened their belief in and commitment to ITI concepts 
and practices. (Contains 14 references.) (SM) 
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INTRODUCTION 



Interdisciplinary Teamed Instruction Background 

The Interdisciplinary Teamed Instruction (ITI) project was a 2-year (1 992-1994) research and 
development project aimed at determining the effects of interdisciplinary teamed instruction on 
teaching and learning in secondary schools. The study also sought to validate the effectiveness of 
a professional development model that would facilitate development, implementation, and evaluation 
of ITI. Through summer institutes and on-site workshops, project staff provided training, resources, 
and ongoing technical assistance to teams of teachers and administrators from four rural Virginia 
secondary schools. These teams designed and tested integrated curricula that incorporated learning 
activities and alternative forms of assessment; engaged in collaborative action research to determine 
the effectiveness of units; and redesigned units using data-based feedback from students, teachers, 
parents, and supervisors (Bums, 1994). 

ITI project participants made two major recommendations: (1) Before ITI is implemented, 
school faculty and community should (a) learn about ITI concepts, processes, and conditions; and 
(b) determine their readiness and desire to begin ITI. (2) AEL should develop resources to support 
ITI implementation (Bums, 1994). 

In response to these recommendations, Rebecca Bums, the ITI project director, wrote 
Dissolving the Boundaries: Planning for Curriculum Integration in Middle and Secondary Schools 
(1995a). This resource, with its Facilitator's Guide (1995b), is a professional development tool 
designed to increase a faculty's understanding of curriculum integration and to assist them with 
decision making about ITI implementation. Since the spring of 1995, AEL staff have conducted 48 
Dissolving the Boundaries professional development sessions in 12 states, as well as the District of 
Columbia and Puerto Rico. Additionally, these resource documents are used by schools, districts, 
and colleges in 42 states, Puerto Rico, Nepal, and Argentina. 

To assist schools that were ready to design and implement ITI, three summer institutes were 
held between June 1995 and August 1996 to help teams of teachers and administrators from 14 
schools (representing five states) build capacity for collaboration and interdisciplinary teamed 
instruction. Two of the institutes were held in 1995: The first was intended to field test the ITI 
concept outside the parameters of a research and development project, and included four schools 
from Virginia, one from Tennessee, and one from Texas. The second was a contracted activity 
conducted for one school in West Virginia. The third institute, held in Virginia in July 1996, is the 
basis for this report. 
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1996 ITI Summer Institute 

The third ITI Summer Institute was held at Radford University in Virginia from July 7-12, 
1 996. Facilitators included an AEL staff member and two consultants. Also in attendance were the 
AEL evaluator and the AEL scaling-up project director. The purpose of the Institute was to help 
participants design integrated units, plan learning activities and assessments, and present their units 
as a culminating activity at the close of the training. 

Planning for the 1 996 ITI Summer Institute focused on recruiting participants from sites 
where ITI had been introduced during the past year. Also, there was an emphasis on recruiting 
interns who might serve as trainers for future ITI institutes and workshops and who would promote 
the use of ITI in their locations. Recruitment was accomplished through personal contact by the ITI 
project leader, by telephone calls, and by mailing a descriptive brochure to other potential attendees. 
One school district that had conducted ITI workshops and one school that had participated in one 
of the 1 995 ITI institutes enrolled teams; another site enrolled two interns. Also, a university faculty 
member was accepted as an intern. The institute was held on the Radford University, Virginia 
campus. 

Thirty participants from four states attended the 1 996 ITI Summer Institute. Four high school 
teams from one county school district in West Virginia attended. These teams (a total of 22 
participants) were part of a group that had participated in an ITI workshop in the spring of 1 996 and 
would continue working with ITI through the 1996-97 school year. Four academic teachers and one 
vocational teacher from the county’s technical education center comprised each team. Two of the 
teams also included a building administrator. A fifth team, from a junior high school in Tennessee, 
was composed of four eighth-grade teachers and their principal. Rounding out the Institute 
membership were three interns. One intern was a local university education department faculty 
member and the other two interns were curriculum specialists from the Oklahoma Department of 
Education. 



Purpose of this Evaluation 

The major purpose of this effort was to conduct a comprehensive evaluation of the 1 996 ITI 
Summer Institute. This evaluation focused on the processes, outcomes, and products of the Institute. 
Key evaluation activities included administering questionnaires on teacher empowerment, schools 
as communities of learners, school effectiveness, and teachers’ sense of control; on-site observations; 
face-to-face interviews with participants; reflective assignments for participants; participant feedback 
forms; oral evaluation feedback provided during the Institute; and review and rating of participants’ 
videotaped presentations. Decisions regarding the evaluations of future ITI summer institutes were 
expected to be informed by the methodology and instrumentation employed in this evaluation. 
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Major Objectives 

The major objectives of this evaluation included the following: 

• to measure participants’ sense of personal locus of control 

• to evaluate the processes of the ITI Institute via participants’ reflective assignments, 
feedback forms, personal interviews, and observations of activities 

• to evaluate participants’ culminating activity of videotaped presentations, completed in 
teams 

• to assess the outcomes of the ITI Institute on participants’ sense of teacher empowerment 
and professional learning community 

• to assess the outcomes of the ITI Institute on participants’ perceptions of their 
organization’s effectiveness. 
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METHODOLOGY 



This section presents the methodology used in completing the comprehensive evaluation of 
the ITI 1 996 Summer Institute. Descriptions of the schedule, the data collection techniques, and the 
evaluation procedures are presented below. 



Institute Schedule 

The Institute was held at Radford University in Virginia, from July 7-12 (see the 
Appendix for a copy of the agenda). Registration and check-in began on Sunday afternoon, July 7, 
followed by an overview session covering introductions, expectations, team roles, team needs, 
networking, and journaling. Each participant received a copy of Dissolving the Boundaries: 
Planning for Curriculum Integration in Middle and Secondary Schools. Participants also received 
notebooks containing additional handouts. 

Monday, July 8, began with an administrators’ breakfast, and then participants were asked 
to complete the “Teacher Questionnaire” and the “School as Community Questionnaire.” This was 
followed by a session on “Developing Teams: Roles, Rules, Relationships, Results.” After a mid- 
morning break, concurrent sessions were held on “Conflict Resolution,” “Tools for Testing 
Consensus,” and “Group Roles and Functions.” 

Following lunch, a “Team Exploration and Debriefing” session was conducted throughout 
the afternoon. This was a unique team-building session, in that each team was to design and 
construct a mousetrap-powered vehicle; these vehicles were then used in videotaped comparative 
trials with other teams. Mousetrap vehicles were constructed from materials such as cardboard, 
styrofoam, wood, plastic, rubber bands, string, cans, lids, etc., and, of course, a mousetrap. In 
addition to team building, the session also introduced participants to the concept of performance 
assessment using a scoring rubric. Another “Team Planning and Reflections” session was held after 
dinner. 



An administrators’ breakfast was again held on Tuesday, July 9, followed by “Five Stages 
of Integration: Reviewing and Rating ITI Examples.” The second morning session was “Discov- 
ering ‘Fertile’ Themes.” After lunch, a “Team Applications: Unit Design” session was held, 
including “Generating Learner Goals” and “Identifying Powerful Learning.” The second afternoon 
session provided team planning time, with topics such as “Beginning the Unit 
Development — Selecting a Theme, Framing Essential Questions, and Designing Powerful Learning 
Experiences.” 

Following the administrators’ breakfast on Wednesday, July 10, participants were asked to 
complete a “Control Questionnaire” and a “School Products and Services Questionnaire.” A short 
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session was held afterward on “Learning Principles and Authentic Tasks.” Afterwards, concurrent 
sessions were held on “Cooperative Learning,” “Problem-Based Learning,” and “Multiple 
Intelligences.” Following the morning break, additional concurrent sessions were held on 
“Cooperative Tasks,” “Community-based Learning,” and “Writing to Learn.” The first afternoon 
session focused on “Structuring a Powerful Learning Experience,” followed by “Team Planning: 
Incorporating Authentic Tasks into Your Unit” and “Team Reflections.” 

Thursday, July 11, again began with an administrators’ breakfast, followed by two morning 
sessions: “Understanding Alternative Assessment” and “Constructing Scoring Rubrics.” After 
lunch, the entire afternoon session was devoted to “Team Planning Time: Construct a Rubric to 
Assess Student Performance, Construct a Rubric to Assess Unit Performance, and Team Reflection.” 
After dinner, teams worked on preparing their team exhibit. 

The last day of the Institute was Friday, July 12, which began with a final administrators’ 
breakfast. Two short morning sessions were held on “Planning for Re-Entry” and “Team Action 
Plans.” After the mid-morning break, team exhibitions were conducted and videotaped. After all 
teams were finished, certificates were presented, and then participants were asked to complete a 
“Feedback Form,” the “Teacher Questionnaire,” and the “School as Community Questionnaire.” 



Data Collection Techniques 

Data collection included a mix of quantitative and qualitative techniques, including four 
surveys, a feedback form, interviews, observations, videotaped presentations, and reflective 
activities. Each method is discussed below. 

Surveys. Four paper and pencil surveys were administered to participants in the 1996 ITI 
Summer Institute. Each survey is discussed in turn. 

Teacher Questionnaire. The first survey was labeled the “Teacher Questionnaire.” This is 
AEL’s adaptation of the School Participant Empowerment Scale (SPES) originally developed by 
Short and Rinehart (1992). The SPES was administered to 4,091 teachers in 183 restructuring 
schools in Ohio by Klecker and Loadman (1996, 1998), who re-factor analyzed the 38 items of the 
SPES into six subscales. All 38 items use the 5-point Likert-type response options of 1 = Strongly 
Disagree to 5 = Strongly Agree. The six subscales and their Alpha internal consistency reliabilities 
were (1) Status, .80; (2) Professional Growth, .70; (3) Self-Efficacy, .89; (4) Decision Making, .80; 
(5) Impact, .83; and (6) Autonomy in Scheduling, .83. The number of items per subscale varied from 
a low of 3 for Autonomy in Scheduling to a high of 12 for Self-Efficacy. In addition to the 38 items, 
AEL staff added eight constructed-response demographic items at the end. These demographic 
questions asked respondents to supply their grades taught, subjects taught, years taught in any school, 
years taught in present school, employed full or part time, degree and credits earned, gender, and age. 
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Table 1 displays the Cronbach Alpha internal consistency reliabilities for all four surveys 
employed in this evaluation. The Teacher Questionnaire Alphas are displayed first in Table 1, for 
each of its three administrations. The full 38-item Alphas are high at .85 and .93, as are the 12-item 
Self-Efficacy subscale Alphas at .88 and .93. The 6-item Status subscale Alphas are .83, .89, and 
.78. Interestingly, the 3-item Autonomous Scheduling subscale has Alphas of .85, .90, and .80. The 
lowest Alphas are .59, .60, and .77 for the Professional Growth subscale. 

School as Community Questionnaire. The second survey was labeled the “School as 
Community Questionnaire”. This survey is AEL’s adaptation of the 22 school-wide professional 
learning community items included in a paper presented by Karen Seashore Louis and Helen Marks 
at the American Educational Research Association (AERA) Annual Meeting in New York City and 
subsequently published (1996a, 1996b). In AEL’s adaptation of items, references to “Since the 
beginning of the school year . . .” were dropped and the response options were placed directly under 
the stem statement. The response option for the stem statements was a 5-point Likert-type scale 
from 1 = Strongly Disagree (SD) to 5 = Strongly Agree (SA). The five subscales in the instrument 
are Shared Sense of Purpose, Collaborative Activity, Collective Focus on Student Learning, 
Deprivatized Practice, and Reflective Dialogue. The number of items per subscale varied from a low 
of 3 items (two scales) to a high of 6 items (two scales). 

Table 1 displays the Cronbach Alpha reliabilities for the total School as Community 
Questionnaire and its five subscales across three administrations. The full 22-item Alphas were .92, 
.93, and .90 for the three administrations, respectively. The 6-item Collaborative Activity subscale 
Alphas were .87, .79, and .84, followed by those for Reflective Dialogue at .89, .87, and .62. The 
4-item Collective Focus on Student Learning subscale Alphas were .60, .70, and .75. As expected, 
the Alphas for the two 3 -item subscales are a little lower, with only the pretest Alpha for the Shared 
Sense of Purpose subscale dipping into the ,50s (.57). 

School Products and Services Questionnaire. The third survey was labeled the “School 
Products and Services Questionnaire”. This is AEL’s adaptation of the “Index of Perceived 
Organizational Effectiveness” (IPOE) instrument originally developed and published by Paul E. Mott 
(1972) and then modified for use in schools by Miskel, Fevurly, and Stewart (1979) and more 
recently by Hoy and Ferguson (1985). The IPOE is an eight-item, self-report instrument in which 
teachers rate the overall effectiveness of their school on four dimensions (quality and quantity of 
product, efficiency, adaptability, and flexibility) with two items for each dimension. Respondents 
select one of five options on a Low (1) to High (5) scale, but the response options/descriptors vary 
by the item stem. For example, for the first item on quantity of production, the responses range from 
“Low Production” to “High Production,” with three other descriptors between them. Possible scores 
range from a low of 8 points to a maximum of 40 points; the higher the score on the IPOE the greater 
the perceived organizational effectiveness of the school. 
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Instrument Name 
Subscale Name 


Number 
of Items 


Alpha Reliabilities 


Pretest 

Administration 


End of 
Institute 


Posttest 

Administration 


Teacher Questionnaire 


38 


.93 


.93 


.85 


Status 


6 


.83 


.89 


.78 


Professional Growth 


4 


.59 


.60 


.77 


Self-Efficacy 


12 


.88 


.88 


.93 


Decision Making 


8 


.78 


.82 


.76 


Impact 


5 


.76 


.82 


.68 


Autonomy in Scheduling 


3 


.85 


.90 


.80 


School as Community Questionnaire 


22 


.92 


.93 


.90 


Shared Sense of Purpose 


3 


.57 


.78 


.86 


Collaborative Activity 


6 


.87 


.79 


.84 


Collective Focus on Student Learning 


4 


.60 


.70 


.75 


Deprivatized Practice 


3 


.65 


.66 


.69 


Reflective Dialogue 


6 


.89 


.87 


.62 


School Products and Services 
Questionnaire 


8 


.85 


N/A 


.91 


Control Questionnaire 


28 


.80 


N/A 


N/A 


Self-Confidence 


14 


.78 


N/A 


N/A 


Autonomous Behavior 


14 


.63 


N/A 


N/A 
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The IPOE has been used in many studies over the years. Early studies reported high reliability 
coefficients; for example, .89 by Miskel, Fevurly, and Stewart (1979). Similar reliability figures 
were reported by Hoy and Fergurson (1985) and, more recently, Loup and Ellett (1996) reported it 
to be .85. Table 1 shows that the School Products and Services Questionaire (IPOE) Alpha 
reliabilities in this study were .85 at pretest and .91 at post test. 

Control Questionnaire. The fourth survey was labeled the “Control Questionnaire”. This survey 
is AEL’s formatting of the “Internal Control Index” instrument developed by Pat Duttweiler ( 1 984). The 
final instrument consists of 28 different statements with one key word missing. The response format 
requires the respondent to choose one of five alternative words to insert into the statement. The five 
alternative words and their associated letters to insert in the blank space are Rarely (A), Occasionally 
(B), Sometimes (C), Frequently (D), and Usually (E). For example, the first item in the instrument 

is: “When faced with a problem I try to forget it.” Per Duttweiler’s goal to eliminate response 

set, 14 of the items are reverse scored and all 28 items fit on one page, printed front and back. 
Duttweiler (1984) completed several reliability and validity studies on the Internal Control Index. 
The internal consistency reliability for the field-test phase (N— 548) was .84 and, for the replication 
study (N = 1 33), it was .85. As part of the validity checks, two factor analyses were completed. The 
first yielded two main subscales of Self-Confidence and Autonomous Behavior, while the second 
produced very similar results. Each subscale consists of 14 of the instrument’s 28 items. 

Table 1 displays the internal consistency reliabilities (Alphas) for the Self-Confidence and 
Autonomous Behavior subscales and total instrument as used in this evaluation (single 
administration only). Datashowsthe 14-item Self-Confidence subscale Alpha to be .78, the 14-item 
Autonomous Behavior subscale Alpha to be .63, and the Alpha for the 28-item total instrument to 
be .80. 



Feedback form. Participants were asked to complete this three-page form at the conclusion 
of the Institute. The first page contained 3 demographic items (name, date, and position), followed 
by 15 closed-ended items, which participants were to complete by using a scale of 1 (Not at all) to 
5 (Very much). The next two pages contained 4 open-ended items each and provided ample space 
for participants’ responses. 

Interviews and observations. The AEL evaluator conducted on-site interviews with 
participants. These structured interviews took place during breaks, lunches, or after sessions were 
completed each day. Each interview consisted of five open-ended questions; responses were 
recorded on summary sheets. The AEL evaluator also attended most of the activities and observed 
both the proceedings and the participants’ reactions. Written notes were generated from this activity, 
and the evaluator met on-site with the presenters during the Institute (days and evenings) to provide 
formative evaluation information. 



erJc 
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Reflective activities. These activities included the “Role- Alike Caucus” and “Team 
Reflection Log”. For the role-alike activity, participants were asked to assemble into one of six role 
groups (administration, social studies, English/language arts, mathematics, science, and other). 
Participants were then given a set amount of time to respond as a group to two reflective questions. 
For the team reflection activity, individual teams were asked to meet briefly following each day’s 
activities (except for the last day) to debrief and think about their needs and concerns and then record 
their thoughts. 

Videotaped presentations. As the culminating activity, each team was asked to make a 
curriculum unit presentation at the conclusion of the Institute, and each presentation was videotaped. 
Criteria for assessing this activity were distributed and discussed with participants during the week. 
The evaluators adopted the Institute’s scoring rubric for their assessment of the teams’ presentations. 



Evaluation Procedures 

This section describes how data collection took place for each of the activities described 
earlier and how resulting data were analyzed. 

Surveys. Each of the four surveys were administered on either Monday or Wednesday of 
the Institute. Participants were given time during the morning sessions to complete the instruments, 
which the AEL evaluator distributed. On Friday, the Teacher Questionnaire and School as 
Community Questionnaire were re-administered to participants during the morning session. Each 
survey was administered to all those participants present at each administration. 

The Teacher Questionnaire, School as Community Questionnaire, and School Products and 
Services Questionnaire were re-administered again in April 1997 to assess participants’ changes 
(growth or decline) in relevant areas. These surveys were mailed to respondents on April 9 (with 
a cover letter and a self-addressed envelope), and a follow-up notice was sent in May. Of the 30 
surveys mailed to participants, 19 usable questionnaires were returned for a return rate of 63%. 
However, not every returned survey had complete data and, thus, the number of matched pairs for 
the t tests of subscales and total instrument scores varied from 14 to 18. 

Using SPSS statistical software program, the AEL evaluator created databases for each of 
the above surveys. Data were entered by AEL support staff, and were then cleaned and verified 
before analyses were conducted. The first step was to compute the internal consistency reliabilities 
(Cronbach Alpha) for each scale and subscale, where appropriate. Next, descriptive statistics were 
computed for each scale and subscale. The t tests by the matched pairs method were computed for 
each applicable pair of scale and subscale scores. For the Teacher Questionnaire and the School as 
Community Questionnaire, two t tests were completed for each subscale and total scale score (pretest 
to end of Institute and pretest to posttest). The alpha level was set at .10. Last, effect sizes were 
computed for each t test and Cohen’s (1977) scheme for effect size descriptors was adopted. 
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Feedback form. The Feedback Form was administered by the AEL evaluator to 29 
participants on Friday morning at the conclusion of the Institute. Again using SPSS, a database was 
created and data were entered. After cleaning, descriptive analyses were conducted for the 1 5 Likert- 
type items on the first page of the survey. For the next two pages of open-ended responses, another 
AEL evaluator read the responses and assigned them to appropriate categories or themes. 

Interviews and observations. The on-site interviews were conducted by the AEL evaluator 
during the Institute. A summary sheet was completed for each of the 29 interviews, and these sheets 
formed the data for subsequent analysis. As well, notes taken for the evaluator’s observations during 
the Institute were reviewed and included in subsequent feedback sessions with the trainers as 
formative evaluation. These oral formative evaluation data are not included in this report. 

Reflective activities. The reflective activities were completed by participants during the 
Institute. Unlike the previous evaluation instruments, which were designed and administered by the 
evaluator, the reflective activities were designed and administered by the Institute staff. Then, the 
completed reflective activity outputs were collected and turned over to the evaluation staff as input 
to this evaluation. The role-alike activity took place early during the Institute, and completed group 
forms were submitted to the presenters. These summary sheets served as data for this analysis. Five 
team reflection logs from Sunday through Thursday also were gathered before the close of the 
Institute for subsequent analysis. 

. ... Videotaped presentations. The participants were asked to make team exhibitions, which 
were videotaped by Institute staff. These videos were reviewed and rated by the AEL evaluators 
using a scoring rubric provided by Institute staff and shared with participants earlier. 
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FINDINGS 



This section presents the findings for each of the data collection techniques used in this 
comprehensive evaluation of the ITI 1996 Summer Institute, preceded by a discussion of the 
characteristics of the participants, gleaned from the demographic questions on three of the four 
surveys. 



ITI Institute Participants 

As stated above, 30 participants from four states attended the 1996 ITI Summer Institute. 
The majority of the participants (22) were from one county in West Virginia. They represented four 
high school teams comprised of four academic teachers and one vocational teacher from the county’s 
technical education center. Two of these West Virginia teams also included a building administrator. 
A fifth team was from a junior high school in Tennessee and included four eighth grade teachers and 
the principal. Three ITI interns participated in the 1996 Summer Institute, one a faculty member 
from Radford University (Virginia) and two were curriculum specialists from the Oklahoma 
Department of Education. 

Of the 30 institute participants, 24 were female and 6 were male. Their ages ranged from 24 
to 62 years with a mode of 46.00 and a mean of 48.43. In terms of their educational level, one third 
possessed a master’s degree plus 45 credits, 7 had a master’s plus 30 credits, 4 had a master’s plus 
60 credits, and 3 had a master’s degree. One participant each reported possessing a bachelor’s 
degree, a bachelor’ s plus 1 5 credits, a bachelor’s plus 45 credits, an educational specialist certificate, 
and a master’s plus 15 credits. All 30 participants reported working full time in their jobs. With 
respect to subjects taught, 5 indicated vocational subjects; 4 each indicated math and/or computer, 
English/language arts, and history/social studies; 3 each indicated combination subjects, other (art, 
music, etc.), and administration; 2 indicated foreign languages; and 1 participant indicated science. 
One additional participant did not respond to this item. 

Participants reported teaching a total of 14 different grade levels and/or combinations of 
grade levels. The 9-12 grade-level group had the largest number of participants (1 1). The 10-12and 
11-12 grade-level groups each had 3 participants in them, while all the others had just 1 or 2 
participants. On balance, the group was very experienced at teaching, with a mean of 22.00 years 
and a maximum of 38.00 years. The minimum years of total teaching experience was 5.00, but the 
median was 22.50 and the mode was 1 9.00. The mean number of years of teaching in their current 
school was 8.57, with a minimum of 1 .00 and a maximum of 22.00. The median years of teaching 
at their current school was 6.00 and the mode was 2.00. 
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Teacher Questionnaire 

Tables 2 and 3 display the results of the three administrations (pretest, end-of-institute, and 
posttest) of the Teacher Questionnaire to the ITI Institute participants. The six subscale results are 
presented in Table 2, while the Teacher Questionnaire Total Score results are presented in Table 3. 
The number of participants with both scores for the t tests varied from 1 6 to 28 for the subscales and 
was 14 and 23 for the Total Score. 

The Status subscale results are displayed first in Table 2* At the end-of-institute 
administration, the mean score declined 0.89 to 22.74. The resultant t value was 2.22, which was 
significant at the .05 level. This is one of only two significant t values in Tables 2 and 3. The effect 
size was -0.247, which is “small” in Cohen’s (1977) terms. The pretest to posttest score also showed 
a small decline (0.28) from 23.72 to 23.44. The t value was a very small 0.28, nowhere close to 
being significant. The effect size of -0.066 was “much less than small.” The professional growth 
pretest mean declined 0.14 at the end-of-institute administration. The resulting t value was very 
small at 0.45, which was not close to being significant. At -0.072, the effect size for this first t test 
was “much less than small.” The pretest to posttest score increased slightly (0.22) to 17.56. 
However, the 0.37 t value was not close to being significant. The 0.107 effect size was “less than 
small.” The self-efficacy subscale mean score in the first t test increased 0.23 to 49.69, while the 
standard deviation decreased to 4.31. The resulting t value of just 0.36 was nowhere close to 
significance. The effect size of 0.047 was “much less than small.” The mean score in the pretest 
to posttest t test increased 1 .38 to 50.13. At 0.97, the /value was not significant, but the 0.253 effect 
size was “small” in Cohen’s terms. 

The Decision Making subscale results are the fourth set of rows in Table 2. In the first / test, 
the mean score increased 1 .38 to 26.08 and the / value was 2.15, significant at the .05 level. This 
is the second significant / value in Tables 2 and 3. At 0.250, the effect size was “small.” The second 
/test increased by 2.00 to 25.59, at 1 .64, the / value was not significant, but the 0.367 effect size was 
“small” and the next to largest such value in Table 2. The first / test reported no change in 
scores — up or down. With no score difference, there was no / value nor effect size. For the second 
/ test on the Impact subscale, there was a 1 . 1 8 increase in the mean score to 1 8 .59 at posttest. The 
1.31/ value was not significant, but the 0.334 effect size was “small.” The Autonomy in Scheduling 
mean score in the first / test increased by 0.59 to 8.85. The 1 .25 / value was not significant, while 
the 0.164 effect size was “less than small.” The second / test showed a 1 .42 increase in the mean. 
At 1 .71, the / value was not significant. However, the pretest to posttest effect size of 0.376 is the 
largest such value in Table 2, although, according to Cohen’s terms, it is labeled “small.” 



♦Due to rounding, slight discrepancies might occur between the Mean and Mean Score Difference 
columns. 
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Table 2: 

Teacher Questionnaire Subscale t Tests and 
Effect Sizes Over Three Administrations 



Administration 

Time 


Number 
of Pairs 


Mean 


Standard 

Deviation 


Standard 
Error of Mean 


t 

Value 


Probability 


Mean Score 
Difference 


Effect 

Size 


Status 


Pretest 




23.63 


3.61 


0.694 












27 








2.22 


.035 a 


-0.89 


-0.247** 


End of Institute 




22.74 


4.13 


0.795 










Pretest 




23.72 


4.23 


0.996 












18 








0.28 


.781 


-0.28 


-0.066* 


Posttest 




23.44 


3.89 


0.821 










Professional Growth 


Pretest 




17.50 


1.93 


0.366 












28 








0.45 


.659 


-0.14 


-0.072* 


End of Institute 




17.36 


1.75 


0.330 










Pretest 




17.33 


2.14 


0.505 












18 








0.37 


.717 


0.22 


0.107* 


Posttest 




17.56 


1.58 


0.372 










Self-Efficacy 


Pretest 




49.46 


4.84 


0.950 












26 








0.36 


.719 


0.23 


0.047* 


End of Institute 




49.69 


4.31 


0.844 










Pretest 




48.75 


5.45 


1.362 












16 








0.97 


.348 


1.38 


0.253** 


Posttest 




50.13 


4.54 


1.136 










Decision Making 


Pretest 




24.69 


5.55 


1.088 












26 








2.15 


.042 a 


1.38 


0.250** 


End of Institute 




26.08 


5.20 


1.020 










Pretest 




23.59 


5.47 


1.326 












17 








1.64 


.120 


2.00 


0.367** 


Posttest 




25.59 


4.78 


1.160 










Impact 


Pretest 




17.15 


3.59 


0.703 












26 








0.00 


1.000 


0.00 


0.000 


End of Institute 




17.15 


3.77 


0.739 










Pretest 




17.41 


3.54 


0.858 












17 








1.31 


.210 


1.18 


0.334** 


Posttest 




18.59 


2.72 


0.659 










Autonomy in Scheduling 


Pretest 




8.26 


3.60 


0.693 












27 








1.25 


.221 


0.59 


0.164* 


End of Institute 




8.85 


3.12 


0.601 










Pretest 




7.71 


3.75 


0.911 












17 








1.71 


.107 


1.42 


0.376** 


Posttest 




9.12 


2.91 


0.706 











* ’"Effect Size = Sma' 



a 




A 




8 



“"Effect Size = Less than small. 



Significant at .05. 
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Table 3 displays the results of the three administrations of the Teacher Questionnaire in 
terms of the Total Score. The possible score could range from 38 to 190 points. For the 23 
participants with complete data, the pretest mean increased 1.48 at the end-of-institute 
administration. At 0.82, the t value was not significant and, at 0.091, the effect size was “much 
less than small.” The pretest to posttest mean score rose 8.43 points to 143.71. The t value of 
1 .65 was not significant. However, the effect size of 0.486, the largest such value for the Teacher 
Questionnaire, was labeled “small” but is very close to the lowest value for an effect size to be 
labeled “medium” by Cohen, which is 0.500. 



Table 3: 

Teacher Questionnaire Total Score t Tests and Effect Sizes 
Over Three Administrations 



Administration 

Time 


Number 
of Pairs 


Mean 


Standard 

Deviation 


Standard 
Error of Mean 


t 

Value 


Probability 


Mean Score 
Difference 


Effect 

Size 


Total Scale Score 


Pretest 


23 


139.39 


16.50 


3.39 


0 .82 


.422 


1.48 


0.091* 


End of Institute 




140.87 


15.42 


3.22 










Pretest 


14 


135.29 


17.34 


4.63 


1.65 


.122 


8.43 


0.486** 


Posttest 




143.71 


11.70 


3.13 











* Effect Size = Less than small. 
* “"Effect Size = Small. 
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School As Community Questionnaire 

Table 4 Displays the results of the three administrations of the School as Community 
Questionnaire to the institute participants. The number of participants with both scores for the t tests 
varied from 18 to 27 for the subscales and was 24 and 18 for the Total Score. 

The Shared Sense of Purpose subscale results are displayed first in Table 4. The end-of- 
institute mean increased 0.22 to 10.1 1. The resultant t value was a low 0.69, which was not close 
to being significant. The effect size was 0.122, which was “less than small.” The pretest to posttest 
scores showed a very small increase of 0.05 to 10.32. The t value was extremely small at 0.09 and 
was close to unity. The effect size of 0.029 was “much less than small.” The Collaborative Activity 
subscale pretest mean increased 2.38 to 17.77 at end-of-institute. This was the largest increase 
across the five subscales in the table. The resulting t value was 3.11, the largest in the table. This 
t value was significant at the .01 level, the only one in the table significant at that level. At 0.534, 
the effect size was “medium” and was the largest in Table 4. The pretest to posttest score increased 
2.11 for the 1 9 participants, which was the second largest increase across the subscales. The t value 
was 1 .86, which was significant at the . 1 0 level. At 0.487, the effect size for the pretest to posttest 
subscale technically was “small,” but is very close to the 0.500 value needed to be labeled “medium” 
by Cohen. The Collective Focus on Student Learning subscale mean score in the first t test increased 
0.34 to 14.96. The resulting t value of 0.63 was not close to being significant and the 0.145 effect 
size was “less than small.” For the 19 participants, their mean score increased 0.53 to 15.84, and 
the resulting t value of 0.70 was not close to being significant. However, the effect size of 0.232 was 
“small” for the pretest to posttest administration. 

The Deprivatized Practice subscale results are the fourth set of rows in Table 4. With a 
possible score of 3 to 15 points, all four scores were less than 7 points. The first t test mean 
increased 0.15 to 6.69 and, at 0.45, the t value was not significant and the 0.058 effect size was 
“much less than small.” Interestingly, the pretest to posttest scores were the same at 6.94, yielding 
no t value or effect size. The Reflective Dialogue mean score in the first t test increased 0.88 from 
18.00 and the t value was 1.51, which was not significant. The effect size was 0.171, which was 
“less than small.” Unexpectedly, the pretest to posttest mean dropped 0.42 to 18.63 (although the 
pretest mean for the 19 participants was the highest for this subscale). Also, both the standard 
deviation and the standard error of the mean decreased at posttest. At 0.39, the t value was very 
small and nonsignificant. The effect size of -0.107 was “less than small.” 
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Table 4: 

School as Community Questionnaire Subscale and 



Total Scale t Tests and Effect Sizes Over Three Administrations 



Administration 


Number 




Standard 


Standard 


t 




Mean Score 


Effect 


Time 


of Pairs 


Mean 


Deviation 


Error of Mean 


Value 


Probability 


Difference 


Size 


Shared Sense of Purpose 


Pretest 


27 


9.89 


1.81 


0.347 


0.69 


.496 


0.22 


0.122* 


End of Institute 




10.11 


2.46 


0.472 










Pretest 


19 


10.26 


2.10 


0.483 


0.09 


.932 


0.05 


0.029* 


Posttest 




10.32 


2.69 


0.617 










Collaborative Activity 


Pretest 


26 


15.38 


4.47 


0.877 


3.11 


.005 a 


2.38 


0.534*** 


End of Institute 




17.77 


4.17 


0.818 










Pretest 


19 


16.00 


4.33 


0.994 


1.86 


.079 c 


2.11 


0.487** 


Posttest 




18.11 


4.37 


1.003 










Collective Focus on Student Learning 


Pretest 


26 


14.62 


2.35 


0.461 


0.63 


.534 


0.34 


0.145* 


End of Institute 




14.96 


2.41 


0.472 










Pretest 


19 


15.32 


2.24 


0.513 


0.70 


.494 


0.53 




Posttest 




15.84 


2.39 


0.548 








0.232** 


Deprivatized Practice 


Pretest 


26 


6.54 


2.58 


0.506 


0.45 


.655 


0.15 


0.058* 


End of Institute 




6.69 


2.54 


0.499 










Pretest 


18 


6.94 


2.90 


0.683 


0.00 


1.000 


0.00 


0.000 


Posttest 




6.94 


2.44 


0.574 










Reflective Dialogue 


Pretest 


26 


18.00 


5.16 


1.012 


1.51 


.143 


0.88 


0.171* 


End of Institute 




18.88 


4.79 


0.939 










Pretest 


19 


19.05 


3.92 


0.900 


0.39 


.698 


-0.42 


-0.107* 


Posttest 




18.63 


3.32 


0.762 










Total Scale Score 


Pretest 


24 


64.83 


13.48 


2.752 


2.09 


.048 b 


3.96 


0.294** 


End of Institute 




68.79 


13.82 


2.821 










Pretest 


18 


67.22 


12.37 


2.916 


0.09 


.927 


-0.44 


-0.036* 


Posttest 




66.78 


17.15 


4.042 











"‘Effect Size=Less than small. """Effect Size=Small. "‘"‘"‘Effect Size=Medium. 
“Significant at .01. b Significant at .05. "Significant at . 1 0. 
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The last set of rows in Table 4 display the results for the Total Score, which could range from 
22 to 110 points. For the 24 participants, the pretest mean increased 3.96 to 68.79. The resultant 
t value was 2.09, which was significant at the .05 level. The effect size of 0.294 was “small.” 
Unexpectedly, the pretest to posttest mean decreased 0.44 to 66.78. The resultant t value was a 
miniscule 0.09 and was close to unity. The effect size of -0.036 was “much less than small.” 



School Products and Services Questionnaire 

Table 5 displays the results of pretest and posttest administrations of the School Products and 
Services Questionnaire. The possible score on this instrument ranged from 8 to 40 and 18 
participants had both scores. The pretest mean was 28.06 and its standard deviation was 4.64. The 
standard error of the mean was 1.095. The posttest mean increased 0.39 to 28.44, whereas the 
standard deviation decreased to 4.30. The standard error of the mean also decreased very slightly. 
The resultant t value was a very small 0.47, which failed to approach significance. The effect size 
was 0.082, much “less than small” in Cohen’s scheme. 



Control Questionnaire 

Table 6 displays the results of the single administration of the Control Questionnaire. The 
Self-Confidence possible subscale score could range from 14 to 70 points. Actual scores ranged 
from a low of 47 points to a high of 66 points. The mode was 6 1 .00 and the median was 59.50. The 
mean was 57.43, with a standard deviation of 6.55. The coefficient of variation (the standard 
deviation divided by the mean) was a low 0. 1 1 4. The standard error of the mean on Self-Confidence 
was 1 . 1 95. The Autonomous Behavior scores ranged from a low of 41 to a high of 65. The modal 
score was 50.00, 1 1 points lower than that for the first subscale. The median was 51.50, 8 points 
lower than that for the first subscale. At 52.03, the mean was 5.40 points less than Self-Confidence. 
The standard deviation was approximately half a point less at 5.92. Interesting, the coefficient of 
variation value was exactly the same as above, although the standard error of the mean was a little 
smaller (1.081). 

The last section of Table 6 shows the Total Scale Score statistics for the Control 
Questionnaire. Here, the possible scale score could range from 28 to 140 points. The actual scores 
ranged from a low of 91 to a high of 131, only 9 points from the maximum possible score. The 
modal score was 93.00, and the median was 111.50. The mean was 109.47, with a standard 
deviation of 10.81. The coefficient of variation was just 0.099, the smallest of the three such values 
in the table. The standard error of the mean was 1 .974, the largest of the three in the table. 





Table 5: 

School Products and Services Questionnaire t Test 
and Effect Size Over Two Administrations 
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Administration 

Time 


Number 
of Pairs 


Mean 


Standard 

Deviation 


Standard 
Error of Mean 


t 

Value 


Probability 


Mean Score 
Difference 


Effect 

Size 


Total Scale Score 


Pretest 


18 


28.06 


4.64 


1.095 


0.47 


.643 


0.39 


0.082* 


Posttest 




28.44 


4.30 


1.014 











♦Effect Size = Less than small. 



Table 6: 

Descriptive Statistics for the Control Questionnaire 



Number 

Respondents 


Minimum 


Maximum 


Mode 


Median 


Mean 


Standard 

Deviation 


Coefficient 
of Variation 


Standard 
Error of Mean 


Self-Confidence 


30 


47.00 


66.00 


61.00 


59.50 


57.43 


6.55 


0.114 


1.195 


Autonomous Behavior 


30 


41.00 


65.00 


50.00 


51.50 


52.03 


5.92 


0.114 


1.081 


Total Scale Score 


30 


91.00 


131.00 


93.00 


111.50 


109.47 


10.81 


0.099 


1.974 




23 
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Feedback Form 

The Summer Institute Feedback Form was administered on the last day to 29 participants: 
23 teachers, 5 administrators, and 1 intern. The overall internal consistency reliability was .74. 

The first 15 items asked respondents to rate the extent to which the Institute had met 15 
separate evaluative topics. Table 7 displays the response option frequencies and percentages, means, 
and standard deviations for all 1 5 items. None of the items received a rating below a 4 on the 5-point 
scale. In fact, 3 of the items (#7, #11, and #12 — all facilitator-related) received consistent ratings 
of 5 from all participants. Similarly, all 15 means were above 4.75, and standard deviations ranged 
only from 0.00 to 0.44. 

Question 16 asked how respondents would describe the Institute experience to others. Out 
of the 41 comments generated, almost half (18, 44%) described it as positive (or similar terms); i.e., 
“A well-planned, carefully orchestrated, flexible workshop.” The next frequent theme was 
motivating/rewarding (8, 22%). In Question 17, participants were asked what ideas, strategies, 
activities, or resources were most meaningful. A total of 76 comments was provided. The most 
frequently mentioned topic was team building (15, 20%), followed by the rubrics (10, 13%). 
Respondents were asked for major strengths of the Institute in Question 1 8. Of the 54 comments 
provided, the most frequently mentioned strength was team building (14, 26%), followed by new 
information/ideas (11, 20%) and organization/implementation (10, 19%). Question #19 asked for 
recommendations for future institutes: 3 1 comments were generated for this question. Almost half 
(13, 42%) focused on adjusting the agenda or schedule. These recommendations ranged from giving 
participants more free time in the evenings to adding more evening activities, as well as adding more 
time for discussion. 

In Question 20, participants were asked how the Institute had changed the way they thought 
about ITI. Of the 28 comments, 8 (29%) indicated a positive change, and 5 participants each (18%) 
mentioned a new belief or reinforcement of ITI ideas and seeing the “big” picture of what ITI 
comprises. Respondents were asked how they would use what they learned in Question 2 1 . Of the 
49 comments, 1 3 (27%) mentioned incorporating ideas and activities (unspecified) in the classroom, 
while another 1 3 focused on team-building with teachers. Eleven of the comments (22%) identified 
specific activities to incorporate, such as rubrics, problem-solving, hats, cooperative learning, 
multiple intelligences, assessment, and journaling. Question 22 asked respondents how AEL could 
continue supporting curriculum integration efforts. Of the 37 comments, the two main suggestions 
were keeping in touch with participants (12, 32%) and providing new information and resources as 
available (11, 30%). Finally, Question 23 asked for other comments. Eleven of the 36 comments 
(31%) were congratulatory. Nine of the comments (25%) mentioned positive aspects of the 
facilities, while 7 (19%) focused on negative aspects. And, 7 comments (19%) were simply “thank- 
you’s.” 
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Table 7: 

Participants’ Responses to 15 Evaluative Topics 
on the End of Institute Feedback Form (N= 29) 



Items 


Response Option Frequencies and Percentages 


Mean 


SD 


Not at all 
1 


2 


3 


4 


Very much 

5 


1 . Had clear outcomes. 


- 


- 


- 


2 

6.9% 


27 

93.1% 


4.93 


0.26 


2. Promoted team building. 




- 


- 


1 

3.4% 


98 

96.6% 


4.97 


0.19 


3. Caused me to reflect on my 
practices. 




- 


- 


4 

13.8% 


25 

86.2% 


4.86 


0.35 


4. Caused me to examine some 
of my attitudes. 


- 


- 


- 


7 

24.1% 


22 

75.9% 


4.76 


0.44 


5. Facilitated development of 
new skills. 


- 




- 


1 

3.4% 


98 

96.6% 


4.97 


0.19 


6. Was relevant to my needs. 




- 


- 


5 

17.2% 


24 

82.8% 


4.83 


0.38 


7. Was conducted in a positive 
manner. 


- 


- 


- 


- 


29 

100.0% 


5.00 


0.00 


8. Had activities that were well 
sequenced.* 


- 


- 


- 


4 

14.3% 


24 

85.7% 


4.86 


0.36 


9. Had activities that rein- 
forced content. 


- 


- 


- 


5 

17.2% 


24 

82.8% 


4.83 


0.38 


10. Included appropriate 
examples. 


- 


- 


- 


4 

13.8% 


25 

86.2% 


4.86 


0.35 


1 1 . Was conducted by 
competent trainers. 


- 


- 


- 


— 


29 

100.0% 


5.00 


0.00 


12. Was conducted in a 
professional manner. 


- 


- 


- 


— 


29 

100.0% 


5.00 


0.00 


13. Had meaningful involve- 
ment of participants. 


- 


- 


- 


2 

6.9% 


27 

93.1% 


4.93 


0.26 


14. Stimulated me to use the 
concepts/skills/materials. 


- 


- 


- 


3 

10.3% 


26 

89.7% 


4.90 


0.31 


15. Provided materials/ideas 
that will be useful to me. 


- 


- 


- 


1 

3.4% 


98 

96.6% 


4.97 


0.19 



*N= 28. 
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Interviews 

The AEL evaluator conducted individual interviews with 29 participants at some point during 
the Institute — during breaks, lunches, or evening hours. The interviewer first asked how much 
decision-making authority the respondent had over what and how material was taught. Seven of the 
29 respondents felt they had total control in deciding what was taught. Six more respondents noted 
they had total control over what was taught, as long as it was within state guidelines/curriculum — while 
5 others perceived they had no control over what was taught, indicating it was based on state 
guidelines. Six others said they had a great deal of freedom in selecting what to teach, and an 
additional 3 said they had limited authority. When reporting how much authority they had in 
deciding how material was taught, the majority (17) said they had total control in this area, 7 said 
they had a great deal of control, 2 said they had limited authority, and 1 indicated no authority in this 
area. The second interview question asked respondents to describe ways they were helping students 
to become independent learners; 46 ways were provided. Critical thinking/analytical skills was most 
frequently mentioned, by 9 of the 29 participants. The second topic was activities and/or projects, 
mentioned by 5 participants. Four respondents each described cooperative learning, individualized 
learning, and computer training. Next, participants were asked how their team members would 
describe students’ learning. Of the 29 respondents, almost half (13) felt their team members would 
say student learning was positive or average. Another 8 felt team members would say that learning 
differs by students, according to their individual abilities. 

For the fourth question, the evaluator asked participants how they would judge their school 
team at collaborating on professional-type issues. Almost half (12) of the participants judged their 
team’s collaboration to be excellent or very good. Seven judged the collaboration to be good. Four 
other participants felt their teams were learning to collaborate. Last, the evaluator asked participants 
to rate their school faculty on readiness and willingness to accept change. Almost half of the 
respondents (13) noted it would be difficult, citing reasons such as “old” faculties, constantly hearing 
about new things, lack of unity, negative attitudes, skepticism, and resistance. Seven participants 
felt about half of their faculty would accept changes, with several noting that their schools had 
implemented so many changes lately that it may impact teachers’ willingness to try anything else. 
Alternatively, 7 other participants felt the majority of their faculty were ready and willing to 
implement school change. 



Role-Alike Caucus Sheet Reflective Activity 

For this activity, participants were to assemble in role-alike groups. The six role groups 
included administration, English/language arts, mathematics, science, social studies, and other 
(including family /consumer science, foreign language, marketing, music, and vocational education). 
For each group, participants were to select a facilitator, a timekeeper, and a recorder, and were to 
answer two questions. 
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The first question was, “What to you seems most promising by way of new approaches to 
teaching and learning?” Looking at responses across roles, five of the groups mentioned learning 
styles and strategies (including higher-level thinking, group learning, questioning techniques, 
cooperative learning, and hands-on learning), and four groups mentioned utilizing new technology. 
Three groups mentioned increasing student interest and participation, and two groups each 
mentioned applying learning to daily life/real world, utilizing school to work, and improving 
communication skills. 

The second question was, “What special opportunities and challenges are associated with 
collaborating with others in designing interdisciplinary studies?” Looking at responses across groups, 
several issues seem to be prevalent. Similar challenges included time restraints, scheduling and 
structure issues, and persuading staff to become risk takers. Similar opportunities included 
collaboration, networking, and sharing resources and ideas. 



Team Reflection Logs 

In this activity, the six teams were asked to complete a Team Reflection Log at the close of 
each day’s sessions (except for the last day). The items for each log were tailored to the content of 
the day’s events. The sixth team, composed of interns, only completed a log for Sunday, which is 
not included in the following analysis. 

Sunday. This daily log asked each team for the following information: Which of today’s 
activities helped set the stage for work at this Institute? What can we do/provide/change to better meet 
your team needs and interests? What are your team’s “next steps” in designing interdisciplinary teamed 
instruction? Four adjectives to describe our feelings. Other things we’d like to say. 

When responding to which of the activities helped set the stage for work at the Institute, 
clearly, the “hands” activity seemed to be the definitive activity to begin the Institute. When asked 
what could be done to better meet team needs and interests, each team had a different response from 
“more time” to “two copies of each handout.” The third item asked teams what their “next steps” 
were in designing interdisciplinary teamed instruction. Responses ranged from Team A, “learning 
how to design and plan a lesson,” to Team D “We don’t know yet — we need to see models of 
successful ITI,”and, Team E mentioned “implement strategies you teach us, find out pilot area for 
integration for our school.” In providing adjectives to describe their feelings, the five teams were 
similar in their feelings of being a little wary of what was taking place, while at the same time trying 
to stay open-minded and positive about the outcome. Finally, when asked for other comments, Team 
A replied, “We are not negative. We are realistic about problems we have no control over to 
implement the program.” Team B asked for “more literal questions, more clarification.” Team C 
said, “This session was a great kick-off. It was well-paced. The leaders are easy to relate 
to/approach.” Similarly, Team D replied, “You are very organized. You seem very 
knowledgeable].” Team E did not provide comments for this item. 
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Monday. Reflective log items for Monday included: Which of today’s activities best helped 
you prepare for your team role and responsibilities at the Institute? What can we do/provide/change 
to better meet your team needs and interests? What are your team’s “next steps” in developing your 
capacity for teamwork? Four verbs to capture our feelings. Other things we’d like to say. 

Two activities that best helped them prepare for their team role and responsibilities elicited 
the six “hats” and mousetrap vehicle construction most often. When asked what could be done to 
better meet team needs and interests, two teams remarked about adjusting the schedule while the 
others were idiosyncratic. In identifying their next steps in developing teamwork capacity, two 
teams mentioned steps to become a better team during the week while the other three teams stated 
aspects that dealt with units/curriculum in the upcoming school year. When asked to list four verbs 
that captured their feelings, analysis of the responses indicates that it seems that all teams were 
actively involved in collaborative development activities. Finally, when asked for other comments, 
responses ranged from “thanks,” “great organization,” to “this is frustrating.” 

Tuesday. The items included: Which oftoday’s activities best prepared you to make decisions 
about interdisciplinary teamed instruction? What can we do/provide/change to better meet your team 
needs and interests? What are your team’s “next steps” in terms of designing an interdisciplinary 
course or unit? Four nouns that sum up our feelings. Other things we have to say. 

In response to the first question, the “fertile themes” article was mentioned most often 
followed by a set of individual responses. When asked what was needed to better meet team needs 
and interests, all responses were positive about the institute and the assistance provided. In 
describing next steps for designing an interdisciplinary course or unit, four teams named adding 
activities to the unit and the other team wrote “children input.” In listing four nouns that summed 
up team feelings, all teams seemed to be “in gear” as the creative process continued. When asked 
for other comments, Teams A and D said they were “beginning to see the light,” while Team B said 
“thanks for the evening off.” Team C noted “we see progress today” and Team E replied, “You, by 
the way you run the workshop into 90 minute blocks of time, have done a wonderful job of modeling 
how to efficiently use the time on block scheduling. You have changed activities often enough to 
keep our interest.” 

Wednesday. These items included: Which of today’s activities best helped you define 
active learning and authentic tasks? What can we do/provide/change to better meet your team needs 
and interests? What are your team’s “next steps” in terms of designing challenging and engaging 
tasks? Four words to describe our feelings today. Other things we have to say. 

When asked which activities best helped them define active learning and authentic tasks, 
three of the five teams mentioned the videotape of classrooms. When asked what was needed to 
better meet team needs and interests, more time was mentioned by two teams and the other responses 
were idiosyncratic. In identifying next steps for designing challenging and engaging tasks, Team A 
mentioned “reflect on theme — revise and refine as needed.” Team B listed “student input, group 
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planning,” while Team C said “fine tune our project.” Team D noted a need to “review closely 
today’s plans to develop the tasks” and Team E said to “design staff development in int. curriculum 
for faculty.” In providing words that described their feelings, responses indicated that, while team 
energy seemed to be waning, each team was close to a final product. Other comments were all 
positive except one that expressed concern about the Friday presentation. 

Thursday. These items included: Which of today’s activities best helped you connect 
learning and assessment? What can we do/provide/change to better meet your team needs and 
interests? What are your team’s “next steps” in terms of building assessment into instruction rather 
than adding it on? Our rubric for today’s program is. Other things we’d like to say. 

When asked which activities best helped the team connect learning and assessment, by their 
responses the rubric activities helped all teams connect the concept of learning with assessment. In 
responding to the item asking what could be done to better meet team needs and interests, two teams 
made positive assertions, two teams asked for copies or a copy machine, and the other team made 
an idiosyncratic remark. In describing next steps for building assessment into instruction (rather than 
adding it on), Team A said, “We want to plan to use more authentic assessment rubrics and less 
traditional methods.” Team B noted, “After administering the initial questionnaire to students in 
Sept., we plan to have students generate guidelines for the interviews of family members.” Team 
C said they would “continue work at home with faculty and other team members [and] try to 
integrate unit with other teachers and integrate the assessment.” Team D’s next steps were to 
“reevaluate task, adjust task, agree.” And, Team E said, “Have students develop their own rubrics 
and use several alternative assessments (w/ rubrics) in each grading period.” When describing their 
rubric, four of the teams seemed delighted with their final rubrics, Team A didn’t seem to be quite 
as pleased. In providing other comments, all teams made positive remarks and three teams included 
a “thanks” or “thankful” comment. 



Videotaped Presentations 

All six teams developed final presentations and presented them on the last morning of the 
Institute. These “culminating exhibitions” were to capture and convey three key concepts: 
(1) Reflect on your learnings and experiences at this Institute; (2) What key messages will you bring 
back to share with colleagues; and (3) How has your work together this week put a mark on how you 
view your role as teacher in the learning process? Teams could select one of the following themes 
for their presentation: TV news broadcast, TV sitcom, home page on the internet, TV commercial, 
TV/radio talk show, magazine, courtroom trial, game show, mystery theater, or broadway musical. 

Each team’s videotaped presentation was reviewed by evaluators and assessed using a 
“Holistic Rubric Based on Clustered Activities.” This rubric included four levels: novice, 
apprentice, practitioner, and expert. 
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The six teams’ culminating exhibitions varied widely in their format, length, and quality, 
although they were uniformly entertaining and obviously fun for both presenters and the audience. 
In terms of the format for the team presentations, two were broadway musicals, one was a TV 
musical (because of the addition of an ITI commercial), one was a TV news broadcast, one was a 
TV shopping network, and one was a TV game show (“The ITI Feud”). The shortest presentation 
was just 2 minutes 5 seconds, while the longest was 13 minutes. All of the presentations included 
some type of visual aid or prop such as handmade signs, pieces of a large puzzle, large maps, 
teaching aids, or overhead transparencies. The presentations varied in their “smoothness,” probably 
reflecting more or less rehearsal time or lack of experience in acting and/or singing. However, the 
chosen format, timing, or quality appeared to not hinder the audience from enjoying some teams’ 
presentations and joining in singing songs or repeating verses when requested. 

Charged with conveying three key concepts in the presentation, the six teams selected a 
variety of concepts, but there was remarkable similarity with one concept. Specifically, the concept 
of multiple intelligences was included in five of the six team presentations, and it dominated the time 
for two of those teams. Most teams used different colored hats to reflect different intelligences; 
either real hats or those made of construction paper or drawn on overhead transparencies. Four 
concepts were mentioned or portrayed in at least two presentations: teamwork, interdisciplinary 
units, assessment/rubrics, and facilitation roles. Other concepts in the presentations included “table 
talk” rules, “hands of traits,” transfer of learning, thinking of teaching and learning differently, and 
elements of a good teaching unit. The evaluators could not identify a third key concept in one team 
presentation, which was the lowest rated exhibition. 

Of the six presentations, two were judged to be at the Expert level, three at the Practitioner 
level, and one at the Apprentice level. Overall, team members seemed to be interested in and 
knowledgeable of the content of the 1996 ITI Summer Institute. 
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CONCLUSIONS 



A number of conclusions can be drawn from the findings presented earlier. These 
conclusions are presented by type of data collection, followed by overall conclusions. 

Teacher Questionnaire. We conclude that the 1996 ITI Summer Institute had some effect 
on the Self-Efficacy, Decision Making, Impact, and Autonomy in Scheduling subscales after a period 
of 1 year. For the Total Scale, the Institute seems to have had some effect, which was almost 
moderate after 1 year. The Institute had little effect on the Status and Professional Growth subscales 
after 1 year. 

School as Community Questionnaire. We conclude that the 1996 ITI Summer Institute had 
a moderate effect on the Collaborative Activity subscale and some effect on the Total Score by the 
end of the Institute. The Institute had some effect on the Collaborative Activity and Collective Focus 
on Student Learning subscales after a period of 1 year. The Institute had little effect on the Shared 
Sense of Purpose and Reflective Dialogue subscales and the Total Score after 1 year. The Institute 
had no effect on the Deprivatized Practice subscale after 1 year. 

School Products and Services Questionnaire. We conclude that the 1996 ITI Summer 
Institute had little effect on the Total Score after a period of 1 year. 

Control Questionnaire. We conclude that the 1996 ITI Summer Institute teachers scored 
higher on the Self-Confidence subscale in comparison to the Autonomous Behavior subscale. And, 
when the subscales were combined, the teachers scored moderately high on the Total Score. 

Feedback Form. We conclude that participants felt the 1996 ITI Summer Institute greatly 
met its intended outcomes. They found the Institute to be well planned, informative, 
motivational/inspirational, and worth the time and money invested. The most meaningful Institute 
activities for participants were team building, assessment rubrics, and multiple intelligences; major 
strengths were team building, information provided, and the organization/ implementation. The 
Institute positively changed participants’ thinking, resulting in increased belief in and reinforcement 
of ITI principles, and heightened awareness of its potential uses. Furthermore, participants were 
eager to incorporate their new knowledge back in their respective schools via rubrics, multiple 
intelligences, cooperative learning, and team building. 

Role-Alike Caucus Activity. We conclude that participants of the 1996 ITI Summer 
Institute felt the most promising new approaches to teaching and learning included technology and 
learning styles and strategies. These styles and strategies included higher-level thinking, questioning 
techniques, group learning, cooperative learning, and hands-on learning. We also conclude that 
collaborating to design interdisciplinary studies brings about opportunities to those involved 




31 



27 



including collaboration, networking, and sharing new resources and ideas with others. And, 
conversely, this collaboration invokes challenges such as time restrictions, scheduling problems, and 
persuading others to become risk takers. 

Interviews. We conclude that most participants of the 1 996 ITI Summer Institute felt a great 
deal of freedom in deciding what and how curriculum is taught in their respective schools. 
Participants also advocated using strategies such as critical thinking and analysis skills, activities and 
projects, cooperative learning, and computer training to help students become independent learners. 
The majority of Institute participants described student learning in their respective schools as 
positive or average, with individual achievement levels differing by student. Most teams felt that 
their level of collaboration was good, if not excellent. About half of the participants felt their 
respective school faculties were ready and willing to implement school change. Another fourth felt 
that about half their faculties were ready at this point, but the remaining fourth felt implementation 
would be difficult at the present time. 

Team Reflection Logs. We conclude that, as the 1996 ITI Summer Institute progressed, 
participants moved from learning and gathering information to clarifying roles and collaborating to 
designing activities to reviewing and revising and, finally, to integrating the unit. As a result, their 
emotions ranged widely from curiosity, apprehension, and anticipation at the beginning to 
enthusiasm and understanding midway through to exhausted, relieved, and productive at the 
conclusion of the Institute. The Institute served as a positive modeling experience for participants, 
and they felt a sense of accomplishment upon mastering assessment rubrics. Further, participants 
felt the most beneficial activities of the Institute included multiple intelligences, rubrics, the 
mousetrap vehicle, and jigsaw. 

Overall Conclusions. We conclude, from the triangulation of the data garnered from the 
mixed-method approach, that the 1996 ITI Summer Institute had the greatest impact on participants 
in the following areas: collaboration, cooperative learning, assessment rubrics, multiple 

intelligences, and team building. Participants were overwhelmingly positive in their valuation of the 
Institute, which strengthened their belief in and commitment to ITI concepts and practices. Finally, 
after reviewing the outcomes of this comprehensive evaluation, we conclude that the quantitative 
instrumentation utilized may not have fully captured and documented institute effects. 



O 

ER i.C 



32 



28 



RECOMMENDATIONS 



Several recommendations are offered, based upon findings and conclusions presented earlier. 

We recommend that the program continue offering ITI summer institutes to interested 
educators. Suggested improvements for future institutes might include more explanatory content, 
more time for individual activities, and slight scheduling adjustments. Also, we recommend that the 
program continue supporting curriculum integration efforts by keeping in touch with participants 
and providing new resources when available. One approach for this communication and information 
provision that is already being implemented by ITI staff is the use of their electronic ITI listserv, 
which connects interested educators from around the world. 

We recommend that the program continue evaluating such institutes, with both qualitative 
and quantitative methods. We do not feel the need, however, for such an intensive, comprehensive 
evaluation effort as presented in this report. The extensive evaluative methodology did not seem to 
be efficacious, in that the amount of information gained did not seem to be worth the prodigious 
amount of time spent gathering and analyzing data. Further, the utility of the three quantitative 
instruments used in this evaluation is questionable, since these surveys did not specifically assess 
the concepts and constructs undergirding the ITI activities and philosophies. We recommend that 
staff locate and secure other relevant instrumentation for future endeavors. And, staff might even 
■ consider alternate ways of focusing evaluative efforts to better document ITI’s ultimate impact on 
student outcomes. Staff might want to investigate in depth why some schools succeed in their ITI 
endeavors and report continuing implementation, while others apparently do not demonstrate similar 
success. One such approach might be site visits at selected ITI schools whose faculty have 
participated in previous summer institutes. 
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Appendix 
Institute Agenda 



ITI Institute Agenda 



Sunday 




Theme: Introductions 




{ Introductions to AEL, the Institute and f acil itators . Getting 
acquainted with our own tearri members , and meeting the other 


teams . ) 




2:00 p.m. - 5: CO p.m. 


Registration and Check- in- -Norwood Hall 


5:00 p.m. - 6:00 p.m. 


Dinner - - Dal ton Hall 


6:00 p.m. - £:30 p.m. 


Welcome _ 

Overview cz anci z,/ip6C 



x I i "tx x o x* —• u. n «- 

Team Rc-ies, Team Heeds, Networking 
Journaling 

(Commonwealth Room, He th : ial i ) 



and 



ERIC 



all working 


sessions are in Heth Hall. 


Monday 






Theme: Team 


Roles and 


Res pons ibilities 


7:30 a . m . " 


8:30 a.m. 


Administrator Breanrast 


8:30 a . m 


10:13 a. re. 


Developing Teams: Roles. Rules, 

Relationships, Results 


10:15 a . m . - 


10:30 a.m 


Break 


10:30 a.m. - 


j_2:00 a.m. 


Tools to Support Teamwork 
Concurrent Sessions: 






Conflict Resolution (Cascades) 

Tools for Testing Consensus (Claytor) 
Group Roles and Functions (New River) 


12:00 p.m. - 


1 p.m. 


Lunch 


1:00 p.m. •• 


4:00 p.m. 


Team Exploration and Debrief 


4:00 p • m . 


5:00 p.m. 


R a R 


5 : 0 0 p . m . 


.-.no 


Dinner- 


' . n r« ... 

O . o w k • 1 ‘ 1 • 


7:30 p - m 


Team Planning and Reflections 


7:30 p.m. - 


r: . O O r~- 

o . i--.ru. 


Ice Cream Social 
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Tuesday 



Theme: Integration Models and Processes 



Designing and developing p-: verms ana ^ p.i 
activities in support of an integrated un.,. 



5 meaningful learning 



7:30 a . m . - 3:30 a . rr. . 

8:30 a.m. -10:15 a.ni. 



10:15 a.m. 

"• C ■ 3 0 a . m . 
12:00 p . m . 
1:00 p.m. ' 



1 0 : <3 0 a.m 

- 1 2 : 0 0 a.m 

- 1:00 p.m. 
2:00 p.m. 



2 : 00 p.m. - .4:00 p.m. 



Administrator Breakfast 

r ive Stages of mtegi at-^on 
Reviewing and Raring IT_ manipxt- ^ 

Break 



Lunch 

T s sit. ?. pp I i c & 1 1 o r: £ • 



Ins me 5 



Ui\ 



Generating Learner Gocils 
Identifying Powerful Learning 

Team Planning Time: 

Beginning the Unit Development 

Selecting a Theme 

Framing Essential Questions ^ 

Powerful Learning mpe 



- l4 s. -i' * jnr; 



4:00 p.m. 
5:00 p.m. 
6:00 P-m. 



C : 0 C p.m. 
- 7:30 p.m 



K * r. 



Dinner 



evening Session 

Videos. Discussion, and Ref lectio: 



ERIC 



Wednesday 

Theme: Building Activities That Link with Assessment 

A well designed activity IS an Assessment- - 1 inking learner 
activities with the culminating performance. 



7:15 a.m. - 8 : 0 C a.m. 

■: . n r, - — - Q * j r . . TTi . 

O . J ri . 1 ; i • y . J. - 

o . 15 r . _ - 20: 3 C £•. . m . 



Administrator Breakfast. 
Learning Principles a 
Concurrent sessions: 



„ j T, . , i- V, G r - — 
llG nJulSliL*'. 



Cooperative Learning > Cl ay tor l 
Problem- Basea Learning (Mew R i > - 

Multiple Intelligences (Cascaces. 
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10:30 a.m. 
10:45 a . m . 



1 0 : 4 5 a . n ; 
12:00 a . m . 



1 2 : 0 C P . it* 

1:00 p . ni . 

2:00 i .a;. 



• 1:00 p . it. 

^ : 00 p.m* 
4:00 t> . rr. . 



4:00 p.m. _ 6 : 3 0 P . m . 
6:30 p.m. - 8:30 p.m. 



t>rea>: 

Concurrent. Sessions: 

Cooperative Tasks (Ciaytor) 
Community ~ bas ed Learning (New River 
Writing to Learn (Cascades) 

Lunch 

Structuring a Powerful Learning 
Experience 

Team Planning: Incorporating Authe 

Tasks into V our unit 

Team F^e flections 

R u R 

Institute Picnic on the Lawn 



Thursday 

Theme: Focusing on Performance 



7 : 


30 a . m . - 


6:30 


a . n\ . 


A dm inist rate r 


8 : 


30 a . m . 


12:00 


p.m. 


Understanding 


Q . 
> • 


30 a . m . - 


- r. . n r. 

i W . -fc. «■' 


a . n*. . 


Constructing 


12 


: 00 p.m. - 


• 1:00 


P . m . 


Lunch 



1:00 p.m. - 4:00 p.m. 



Team. Planning Time: 

Construct a Rubric to 
Assess Student Performance 
Construct a Rubric to Assess Unit 
Performance 
Team Reflection 



o 

o 


p.m. 


- 5:00 


p . m 


o 

o 

LO 


P . m . 


- 6:00 


p . m 


6: OC 


. P . m . 


!,as ne 


.r. .z. 



F: Sc P. 

Dinner 

Team Planning: Prepare Team E:-:nib 



ERIC 
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Friday 

Theme: Culminating Performances 



7:30 a . m . - 


S : 30 


a.m. 


Administrator Breakfast: 


8:30 a . m . _ 


9 : 00 


a.m. 


Planning for Re-entry 


9:00 a.m. - 


9 : 30 


a.m. 


Team Action Plans 


9:30 a.m. 


10 : 1 


5 a.m. 


Team Exhibitions 


10 : 15 a.m.’ 


- 10: 


30 a.m. 


Break 


10:30 a.m. 




15 a . rn . 


Team Exhibitions (continued) 


11:1: a.m. 


* ^ | 


C L' r . ii« . 


Presentation of Certificates 



Program' lv U ation 
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